Performance Metrics and Evaluation Issues for Continuous Activity Recognition

In this paper we examine several factors that influence the evaluation of multi-class, continuous activity recognition. Currently, there is no standard metric for evaluating and comparing such systems, although many possible error formulations and performance metrics could be adapted from other domains. In order to make progress toward a standard metric appropriate for evaluating activity recognition, we outline the sources of errors in such systems, present different methods for detecting and labeling these errors, and compare existing metrics with a more nuanced performance visualization. We conclude with a discussion concerning the interpretation of the visualization for comparing recognition systems in different domains.